{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<div class=\"jumbotron\">\n",
    "    <h1 class=\"display-1\">人工智能与深度学习</h1>\n",
    "    <hr class=\"my-4\">\n",
    "    <p>主讲：李岩</p>\n",
    "    <p>管理学院</p>\n",
    "    <p>liyan@cumtb.edu.cn</p>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "> ... human needs to understand how a particular algorithmic decision has been arrived at"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 引子"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<center><img src=\"./img/1_introduction/nobelPhysics2024.jpg\" width=100%></center>\n",
    "\n",
    ">John Hopfield (left) and Geoffrey Hinton (right) won the 2024 Nobel Prize in physics for discoveries that allow machine learning with artificial neural networks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- John Hopfield，普林斯顿大学\n",
    "    - 设计了早期的神经网络——Hopfield网络\n",
    "\n",
    "<center><img src=\"./img/1_introduction/Discrete-Hopfiled-Network.png\" width=60%></center>\n",
    "\n",
    "- 循环网络(recurrent network)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Geoffrey Hinton, 多伦多大学\n",
    "    - 基于Hopfield网络设计了被称作玻尔兹曼机（Boltzmann machine）的神经网络\n",
    "    - 包含隐藏神经元\n",
    "    - 提出反向传播算法\n",
    "    \n",
    "    > [Learning representations by back-propagating errors](https://www.nature.com/articles/323533a0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- AI在业界快速发展"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<center><img src=\"./img/1_introduction/AIhardware.png\" width=60%></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<center><img src=\"./img/1_introduction/AITop5Stocks.webp\" width=80%></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<center><img src=\"./img/1_introduction/AISupplyChian.png\" width=60%></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 从自动化到智能化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "### 自动化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 自动化：将人类的业务逻辑转变为能够自动运行的应用程序\n",
    "    - 具有形式化的规则"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 例如，MRP、ERP、RPA (Robotic Process Automation)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 可以解决对人类来说特别复杂，但对计算机而言很容易的问题"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 例如，象棋、五子棋、数独游戏等"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- 但，另外一些问题则不易被形式化规则所描述"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 例如，接受地理信息、卫星图像和一些历史天气信息，预测明天的天气\n",
    "- 理解自然文本表示的问题，正确回答该问题\n",
    "- 接受一张图像，识别出该图像所包含的人，并在每个人周围绘制轮廓"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 上述这些问题的共性：很难写出完美的形式化规则"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "\\begin{problem}\\label{prob:imperfectrules}\n",
    "如何解决上述问题？\n",
    "\\end{problem}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### 表示"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 一种方式用形式化的语言将知识硬编码（hard-code），计算机利用逻辑推理规则自动理解形式化语言\n",
    "    - 人工智能知识库（knowledge base）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 最著名的人工智能知识库是Cyc (名称来源于encyclopedia)项目，包括\n",
    "    - 一个推理引擎\n",
    "    - 由CycL语言描述的声明数据库，由人输入完成\n",
    "    - [https://cyc.com/platform/](https://cyc.com/platform/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- Cyc推理引擎\n",
    "    - 组成\n",
    "        - 1.5 million concepts\n",
    "        - 25 million rules\n",
    "        - 1000多个子模块\n",
    "    - 功能\n",
    "        - 利用常规（common-sense）知识和深度领域（deep domain）知识\n",
    "        - 做出上千步的推理"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 但，Cyc不能理解人在早上剃胡须的事情"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 硬编码的失败表明计算机需要具备从原始数据中提取模式的能力（即自己获取知识的能力）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- 机器学习（machine learning，ML）是一类强大的可以从经验中学习的技术。 通常采用观测数据或与环境交互的形式，机器学习算法会积累更多的经验，其性能也会逐步提高"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<center><img src=\"./img/1_introduction/wake-word.svg\" width=80%></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 机器学习算法性能取决于**对给定数据的表示**（representation)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<center><img src=\"./img/1_introduction/cartesian2polar.png\" width=60%></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 在笛卡尔坐标系下很难区分的两类数，转换到极坐标下就很容易区分开来"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### 机器学习组件"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 数据\n",
    "    - 特征（feature）\n",
    "    - 维度（dimension）：特征向量的长度\n",
    "    - 标签（label）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 目标函数（objective function）：模型优劣程度的度量\n",
    "    - 可优化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 算法：搜索出最优的模型参数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### 表示学习"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 很多任务难以确定提取哪些特征\n",
    "    - 例如，识别照片中的汽车"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 解决方式之一是用机器学习发掘表示，即**表示学习**（representation learning）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 设计特征或设计用于学习特征的算法，目标通常是识别出影响观察数据的因素"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 但在实践中表示学习的困难在于，多个因素同时影响观察数据，因此，从数据中直接提取高层次、抽象的特征是非常困难的"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- 解决方案：**深度学习**（deep learning），用简单的表示来表达复杂的表示"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<center><img src=\"./img/1_introduction/deeplearningRepresentation.png\" width=60%></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 深度学习将复杂映射分解为一系列嵌套的简单映射，解决表示学习的难点"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- 深度学习是通向人工智能的主要途径之一"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<center><img src=\"./img/1_introduction/AIvenn.png\" width=50%></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- 不同类型系统的组成"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<center><img src=\"./img/1_introduction/components.jpg\" width=60%></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 深度学习的发展"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "### 深度学习的三次浪潮"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<center><img src=\"./img/1_introduction/AIHistory.webp\" width=100%></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<center><img src=\"./img/1_introduction/deeplearningWaves.png\" width=60%></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 第一次浪潮始于1940年代的兴起的控制论"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 第二次浪潮始于1980年代的联结主义和人工神经网络"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 联结主义：当网络将大量简单的计算单元连接在一起的时候可以实现智能行为"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 反向传播算法\n",
    "- 序列建模方向的进展：长短期记忆模型（LSTM）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- 第三次浪潮始于2006年"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 深度信念网络出现带来的突破"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 深度学习模型开始展现优于其他机器学习技术和手工设计的AI系统"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### 深度学习模型的演进"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 数据集vs计算机内存和计算能力\n",
    "\n",
    "| 年代 | 数据规模              | 内存   | 每秒浮点运算        |\n",
    "| :--- | :--- | :--- | :--- |\n",
    "| 1970 | 100 （鸢尾花卉）      | 1 KB   | 100 KF (Intel 8080) |\n",
    "| 1980 | 1 K （波士顿房价）    | 100 KB | 1 MF (Intel 80186)  |\n",
    "| 1990 | 10 K （光学字符识别） | 10 MB  | 10 MF (Intel 80486) |\n",
    "| 2000 | 10 M （网页）         | 100 MB | 1 GF (Intel Core)   |\n",
    "| 2010 | 10 G （广告）         | 1 GB   | 1 TF (Nvidia C2050) |\n",
    "| 2020 | 1 T （社交网络）      | 100 GB | 1 PF (Nvidia DGX-2) |\n",
    "\n",
    "- FLOPS（float operations per second）：每秒浮点运算次数\n",
    "    - KF：kilo FLOPS，每秒一千次\n",
    "    - MF：mega FLOPS，每秒一百万次\n",
    "    - GF：giga FLOPS，每秒十亿次\n",
    "    - TF：tera FLOPS，每秒一万亿次\n",
    "    - PF：peta FLOPS，每秒一百亿亿次"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<center><img src=\"./img/1_introduction/AIenergy.png\" width=100%></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- ImageNet竞赛\n",
    "    - [https://www.image-net.org/challenges/LSVRC/](https://www.image-net.org/challenges/LSVRC/)\n",
    "    - Dr. Feifei Li (李飞飞)\n",
    "    \n",
    "    <img src=\"./img/1_introduction/feifeili.png\" width=60%></img>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- ImageNet的图片\n",
    "\n",
    "<center><img src=\"./img/1_introduction/imagenetSample.gif\" width=100%></center>\n",
    "\n",
    "- 图片规模\n",
    "    - 目前共有14,197,122幅图像\n",
    "    - 21,841类别(synsets)\n",
    "- 竞赛场景\n",
    "    - 图像分类\n",
    "    - 目标定位\n",
    "    - 目标检测"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<center><img src=\"./img/1_introduction/imagenet.png\" width=60%></center>\n",
    "\n",
    "- 人眼辨识图片的错误率约为5.1%"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### 神经网络模型的关联\n",
    "\n",
    "<center><img src=\"./img/1_introduction/neural_networks_evolution.png\" width=80%></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- FFNN，Feed Forward Neural Network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- RNN, Recurrent Neural Netork\n",
    "- LSTM, Long Short-Term Memory 长短期记忆模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- CNN, Convolutional Neural Network 卷积神经网络"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- GAN, Generative Adversarial Networks 生成式对抗网络"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Transformers\n",
    "    - sel-attention mechanism 注意力机制\n",
    "    - BERT, Bidirectional Encoder Representations from Transformers\n",
    "    - 适用于自然语言处理"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- LLM, Large Language Model\n",
    "    - 多模态（图像、声音、文本等）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### 深度学习的应用领域"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 智能助理\n",
    "- 翻译\n",
    "- 图像识别\n",
    "- 自动驾驶\n",
    "- 游戏"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "[https://d1m75rqqgidzqn.cloudfront.net/wp-data/2021/03/15175845/April-28-deep-learning-applications-infograph.png](https://d1m75rqqgidzqn.cloudfront.net/wp-data/2021/03/15175845/April-28-deep-learning-applications-infograph.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- Can LLMs be Scammed? A Baseline Measurement Study [doi.org/npkh](doi.org/npkh)\n",
    "\n",
    "<center><img src=\"./img/1_introduction/AIScammed.png\" width=80%></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- AI-Edited Images and Videos Can Implant False Memories and Distort Recollection [doi.org/nj2k]\n",
    "\n",
    "<center><img src=\"./img/1_introduction/AIPhoto.png\" width=80%></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 课程与教材"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 教材：《动手学深度学习》\n",
    "    - [https://zh-v2.d2l.ai/](https://zh-v2.d2l.ai/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 考核：\n",
    "    - 出勤（10\\%）\n",
    "    - 实验（30\\%）\n",
    "    - 考试（60\\%）"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "幻灯片",
  "hide_input": false,
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autoclose": true,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 1,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": false,
   "user_envs_cfg": false
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
